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A 9-item questionnaire was given to registering students for fall and spring 
1969. For easier handling! its form was changed for the next semester. By exxxJent, 
the two versions were handed out and 687 students answered both in quick 
succession. The staff decided to compare the two forms to see if the answers 
differed. Two questions were considered unsuitable for comparison. For the other 
seven, the frequency of differing replies ranged from 16 to 667. Three po^ible 
explanations were offered: (1) students do not take such questionnaires seriously. <Z) 
they are unable to answer reliably. (3) angry at answering the same questions twice, 
they deliberately mis-answered the second time. If the first or second is right, such 
data gathering is a waste of time. If the third is right, the amount of data gathering 
should be reduced, assuming that the more questions a student must answer ..the less 
reliable the replies. Point 3 will be re-examined in the fall 1969 semester, when bom 
versions of the questionnaire will be given out at an interval of a few weeks. It is 
expected that, with the lapse of time, the student will feel no anger and may even 
forget he answered before. If marked dif ferences in replies still show up, the matter 
should be pursued. While the format of a questionnaire does affect the way it is 
answered and may explain the differences found in this case, the value the 
instruments is doubtful— for there is no way of knowing which format, if either, eboteo 
reliable data. (HH) 
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This report documents the results of administering the same questions 
twice to the same respondents within a few minutes. The questionnaires used 
and the procedures followed are part of data gathering activities exercised 
by Project Follow-Through, a project sponsored in part by the California 
Stete Depart sent of Education. 

P ITO M 4 \ !T * ® 

Each semester, when enrolling for classes during the coming semester, 
students of the Orange Coast Junior College District are asked to complete 
an enrollment questionnaire as part of the semester registration procedure. 
Figure I shows two versions of the enrollment questionnaire. The upper 
version represents the form used during the Fall and Spring registration, 
1968-69. After using this form, we decided, for ease in keypunching, to 
convert the form to a vertical format as illustrated in the lower version. 
This new version was used in the Summer, 1969, registration. 

During the registration procedure for Summer, 1969, some of the old 
version enrollment questionnaires inadvertently found their way to the 
registration line. There, doubtless because of their different appearance, 
they were given to students registering along with the new version. As a 
result, 687 students responded to both versions of the enrollment 
questionnaire. It^is doubtful that more than a few minutes elapsed between 
responding to the two questionnaires inasmuch as nearly all students complete 
the registration process with one visit to the place of registration. 

This phenomenon was discovered while processing the questionnaires and 
it occurred to us that it would be interesting to count up the number of 
times each student answered the same question differently. This we did. 

The results, we think, are startling. 
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Results 



Figure II shows a tabulation of responses to questions found in the 
enrollment questionnaires returned by the 687 students who answered both 
versions. The tabulation shows for all questions, save 4 and 6, the number 



when answering the same question on both forms, answered it differently. 
Questionnaire item 4 is excluded from the tabulation because the question 
itself is not the same on both versions of the questionnaire. Question 6 is 
eliminated from the tabulation because it asks the respondent to look up a 
number in a table of major codes. We judged this to be so error prone that 
little, if anything, could be learned by comparing the different responses 
between versions of the questionnaire. 

The frequency with which students answered the same question differently 
is striking, if not frightening. They range, as a percentage of 687, from 
16 to 66. We have not measured these differences in terms of statistical 
significance mainly because the responses to the two questionnaire versions 
can hardly be construed as independent. The .magnitude, however, ol: the 
number of differences makes us believe intuitively, for what it's worth, 
that they are meaningful; 

Conclusions 

Although we generally forbear offering conclusions based upon the 
statistics produced in the execution of Project Follow-Through, we can't 
resist acknowledging a few self-evident features of this casual study. At 
least three explanations come to mind with respect to the large number of 
different responses we find in this comparison: 



of students who answered the 




who auswered the question vsi the second 




ERIC 



3 



1. Students do not take data gathering questionnaires seriously. 

2. Students take data gathering questionnaires seriously, but are 
unable to answer questions reliably. 

3. Students answering both versions of the enrollment questionnaire 
became angry at answering the same questions more than once and 
deliberately mis-answered the questions. 

If 1 or 2 of the above prevail (and it doesn't really setter which), 
then we are ready to conclude that gathering data from students through the 
use of questionnaires, application forms, and similar techniques is so 
unreliable as to render the activity a waste of time. 

If 3 prevails, it behooves us to take strenuous steps to clear 
registration procedures of redundant and unnecessary data gathering attempts. 
There is reason to believe that the more questions a registering student must 
answer, the less likely he is to answer them reliably. 

We intend to examine point 3 more thoroughly. If students answering 
both versions became angry at answering questions twice and reacted in a 
recalcitrant fashion, then their answers should be more reliable if they were 
not angry when answering the questionnaire the second time. 

During the registration for the Fall Semester, 1969-70, we will take 
advantage of an opportunity to issue both versions of the enrollment 
questionnaire to the same students at times separated by several weeks. The 
time lapse, we hope, will cool the respondents ire. Too, he may forget he 
answered the questions before. 

A random sample of students will be given a different version of the 
enrollment questionnaire to answer for a second time when they return to 
confirm their summer-time registration shortly before school starts in the 
fall. If marked differences still prevail in the ways that they answer the 
questions, then w«; might suspect that anger ie not the key and we should look 




elsewhere. 
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We are mindful of the important effects that questionnaire formats have 
upon the manner in which questions are answered. If the differences we have 
found with our enrollment questionnaires can be explained away as a result 
of format differences, then we are again ready to discount heavily the use 
of questionnaires and such devices as reliable data gathering instruments 
After all, who is to say which oi several formats will gather accurate data 
and which will not? 
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PROJECT FOLLOW-THROUGH, SEMESTER ENROLLMENT QUESTION NAIR E 



. Ksr.ie. 



. I.D. No. 



If yoo or* em p loyed, how «My hours weekly do yov work? Q 0 Q MO 
2. Approximately how much do you oom por wock while attending college? $_ 



Soc. Sec. New. 

□ 11-20 □ 2140 



G 31*40 Q 40 Plus 



3. Are you now seeking part-time employment (present job status not considered)? O Yes O No 

4. If you are .xsw working (or in the armed services) 30 or more hours per w.elc, do you work 

O In your ma|or field □ In an unrelated field Q In a related field Q Working leu 1 

5. What ore your transfer plans at this time? □ Non-tronsfer □ California State College 

O Private California College or University Q Out-of-State College or University 

6. Write your major code here (see reverse side) - 

7. Will you complete yeur Jorior =!':;= program this semester? □ Yes Q Ho 

8. At this timo what Is yeur career obiective? 



■ 30 

O CaOfondo 



University 



□ Professional (five or mere years of college) 

( 1 fttO teAos e ka t «• »*««^ — — 



Q Educator lj Miiiiary Q Technician 

G roe An* G =r Jryrr'eyman p Self Employment Q Other 

9. On what basis have you cheeen your career objectve? O Parent's or guerdon's advice Q Friend's, adviee 



□ Athlete 

H Undecided 



□ College counselor's advice 

□ Your own determination 



D College instructor's advice Q Advice of high school 

O Other Q Und -cidsd on career objective 
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Figure X 
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N = 687 



Figure II 





Duplicate Semester Enrollment Questionnaires 




QUESTION NO. 


ANSWERED-A 


ANSWERED-B 


DIFFfeREN't ANSWERS 








Total 


Percent Of 687 


i 


632 


578 


398 


58 


2 


483 


348 


450 


66 


3 


650 


650 


238 


35 


5 


672 


666 


213 


31 


7 


655 


664 


110 


16 


8 


671 


666 


317 


46 


9 


678 


648 


284 


41 



